A General Approach to Sparse Basis Selection: Majorization, Concavity, and Affine Scaling

Measures for sparse best–basis selection are analyzed and shown to fit into a general framework based on majorization, Schur-concavity, and concavity. This framework facilitates the analysis of algorithm performance and clarifies the relationships between existing proposed concentration measures useful for sparse basis selection. It also allows one to define new concentration measures, and several general classes of measures are proposed and analyzed in this paper. Admissible measures are given by the Schur-concave functions, which are the class of functions consistent with the so-called Lorentz ordering (a partial ordering on vectors also known as majorization). In particular, concave functions form an important subclass of the Schur-concave functions which attain their minima at sparse solutions to the best basis selection problem. A general affine scaling optimization algorithm obtained from a special factorization of the gradient function is developed and proved to converge to a sparse solution for measures chosen from within this subclass.